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TEST SCORES, COURSE GRADES i AND BOTTOM LINES 



Because I linplcmont t\\o required wr1t1nrj proflciGndy eMainihations for 
thfdo of the coHgocs of the Univorslty of Wisconsin-Mi IwaukeQ whilG coordlnot- 
1ng eight elective wiHtlng coui-rins, I rilso d)(pla1ni justify, and occasionany 
defend progr^anniiatic , course, oi'^ individual evaluations of writing. And because 
1 do soi I ani IncrDaslngly dlsqulotod by our^ discipHne's coalGSCing approach to 
valid and reliable evaluation, The news that Johnny couldn't writo was 
followedi appropriately 5 by a rash of testing programs* an NOTE resolution 
about the nature of supportable testsi and oonie hard questions among ourselves 
about what wq mean when we say that students write either well or poorly. In 
the past year two publications, O'Dell and Cooper's lyal^atjn^g Writing.^ and the 
SprinOf 1978, Issue of Basic Writi ng., have even further lad programs and indi- 
vidual teachers toward new methods of evaluation. We have now what someone 
reared on "write about the parking situation and he'll flunk you" methods of 
grading might take for a complete inventory of trustworthy^ fair methodologies* 

What I findi however, is that although we have new wordsi the music re- 
mains the same^ the hard questions that are raised about writing evaluations 
linger on. We still have no theory of evaluation, no philosophy of interpreta- 
tion! in which to embed the results of our tests* Whether working as 
researchers 1n large programs or as tBachers explaining a grade on a paper to 
one student 5 we have no way to connect nnost of the newly valid and reliable 
results of evaluation to the rhetorical situation of an author and a reader. 
Without a theory I we unknowing! / Invite attack, 'and will, paradoxically by 
virtue of our new methods, continue to do so unless we question and diversify 
their results , 
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For instance, aditil ni fitrators of progrfltiis, poople in the situation 1 
faced whon trying to convincfj Ohio State University to begin an exponsivc but 
essontial reniedial program for those 2,000 open adinissions students who 
scored bolow IB on the English ACT test each yeari niay nttcnipt, as 1 did, to 
show that students ncGd additional instruction hy aliovnng that they cunnot 
benefit from availablQ Instfuction. The niethod adopted in such situations » 
demonstrating that trained readers of anonymous samplGs agree that a laroo 
percentage of student vvriters cannot achieve an agretid-upon minimal standard, 
may (as it was at Ohio State) be successful. And then questions about prior* 
and continuing methods of evaluation begin t 

When a remedial program required for 1/4 of the students begins, why is 
there no rise in the grade distribution for the regular freshman course, even 
when course standards and objoctivos remain the same? Why were 1/4 of the fresh 
man students required to take remedial courses if 40% of the class i^ere already 
getting A's and B's in required freshman English? Why is there, in sumi no 
discernible relationship between a teacher's judgment about a course grade and 
valid and reliable holistic measurements of writing samples? 

In a second context, we receive attacks that our demonstrations in the 
name of program development do not Invite. For instance, what does a teacher 
reply to Rosemary Hake and Joseph William's Chicago demonstration that although 
English teachers trained in holistic reading teach in class that good writing 
has a verbal (active) style, they actually rate higher papers written in the 
less readable but more ponderous nominal style associated with bureaucrats and 
academic research? 

Or, in yet a third context, what do we say about the recent research 
showing that essays written by recogn1?ed professional writers inserted Into 
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Stacks of student essays on the %(\m topic were JucUjcd by (again) ty^tilnGd 
lioHstic faters to bd p&orcr than tho student thomcs? (low do wc make this 
r^GsUU cOrigruent with Kelldg llurlt's standard oViiludtlon theit cldar^ly pUidOs 
tha syntfldtic maturity of focpgriUcd prdfosBional wrltors "two stops abovia 
Ijth graders? If wo ''know- thnt professionals wr1to a inoro syntactlcany 
mature prose than studdntsi would we ever credit even trained teachers* ratings 
of and in this cqsg prefordhcc for student papers? Without rejecting syntactic 
maturity as a devclopnicn tal goal ^ how niight we explain the teachers' 
preferences? 

And in a fourth, final, Inimediatfi, and perhaps most important conteKt» 
what do v^e tell a student who correctly identified the uso of the eMclarnatiOn 
point on an objective test, but whose teachers has just circled all of his 
definitely correct uses of that mark on an essay for class? No one told the 
student that the most important rule about the e)^clamation mark 1S| in 
practice, to use it sparingly; someone lowered his grade. Students who do 
learn handbook rules about punctuationj diction, or usage may thinks justi- 
fiably, that our class evaluations of their writing are rarely rule-governed. 

We need to reconsider our Interpretations of such judgfnentSi especiany 
of valid and reliable judgments^ 1f we are to confront these persistent ques- 
tions. The worst is true, Even with, or perhaps especially with^ valid and 
reliable methods of evaluation, we appear to contradict ourselves. We wave 
Strunk and White at students' essays and give high grades for prose they 
despised^ We appear to yrade writing in class on the basis of unreliable or 
invalid procedures* We only believe In exclamation points when 1t suits us to. 
Me beg for money to instruct students we at one minute call rernedlal and in the 
next give A's and B*s. 
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A co\mmt theot^y of intm^prctatioii stanils up to thcso critictsins a$ 
rn)\Anm on methods or tools of evaluation cannot. Our tcchnolody-tha 
homs, statistical veH flcationp , bar gfapHs, iimtrlces, priiiiary fGatufGS, 
and felativo reatlablllty Indi'cos now available to apply to studont wHting- 
lags behind our philosophy, our roasonod rhetorical explanation of tost 
results. In order to show how far these tools lag behind our noeds, 1t3t me 
r-evlew tho principles and proirlses that inforni them: 

1. Evaluations of writing, whether they Judge or describe, are 

measurement. 

2. Measurcnients are always and only rGlative. 

3. Formal, accepted iiieasureinents of writing may indicato quality 
relative to two percoptiblQ variables: time, and frequency. Tinie elements 
may be identiflGd and measured: the time 1n which a piece was written, the 
time It took to read the piece, the grade-level or age of the writer, the 
amount of instructional time given before or after the writing sample or 
other test was assigned, the number of yoars experience of the teacher, the 
politics of the particular decade in which the same was evaluated-all of 
these temporal elements may apply In evaluations of writing. 

4. The frequency of the grammatical, syntactic, stylistic, or or- 
ganizational markers within a piece of discourse are the other quantifiable 
variables measured by standard evaluative methods. The per/discourse unit 
numbers of errors, of t-umts, of propositions, of generalizations, of 
transitions, of abstractions, of various embeddings, of allusions, of 

metaphors--all may be counted. 

5. Currently touted metHods of evaluating writing may aim to predict 
or count these temporal or frequency-related characteristics of writing in 
order to define one of the following: 
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1. Changes 1n student w>^1t;tna (alther by IncUvklunls 
or groups) rol.itlve to time', 

2. ApproprlatG instructional placomont relatlwe to 
froquoncy coiints In the vriting of othor students, and 

3. AchiGVcmont rGlative to previously defiried syntactic, 
course porrormancQ, mlninial cofflpDtency» or proficiency 
standards . 

The key term In each of those adiiilsslble alms of ineasurcmcnt , is, of 
course, "relative." Our toolfs evaluatQ writing 1n relationship to only time 
and frequency, oven holistic readings judge writing relative to the frequency, 
related itupfessions of roadors of one population of essays, and the rate of 
reader agreement in such evaluations is carefully controlled by training 
readers about what to look for. 

This sort of evaluative technology, however it may be validated, wars 
with the processes of reading and writing a text. I would be the last person 
to suggest that whimsical, subjective responses to student writing should 
once more reign with elitist arbitrariness-, "I just don't like it" is not, I 
know, an aiternatlve to placement or proficiency based on mean scores and 
controlled test situations. Nonetheless, the results of newly controlled 
evaluations of writing are, as now described and practiced, Inadequate unless 
they are Interpreted w1th1n a rhetorical context. We are embracing evalu- 
ative tools that by their nature must ignore the situation of the reader or 
of the writer, and in doing so we are newly opening ourselves to warped methods 
of instruction tied to such evaluations. 

To demonstrate what I mean, let us return to the recent research about 
trained readers' preferences for student over professional writing and try to 
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construe possible Intov^p^^ecallons of this 1nve5t1gat1on. Toacliofs werQ tralnocl 
to fead pape»*s holistlcfllly In orddr to rota them against othev^ papers 1n a 
sdinple populatldhi Wh^n thosu teachfifs then ratod student writers higher thah 
professional writers wt^itiny about the saine topic i a numbor of conclusions 
itilght bd drawn* It inlght bo possible, for Gj^ainplo, to say that tGnchors can't 
judge the quality of prose , even after training, Or to say that teachers prefor 
youthful points of view on any topic* Or to say that any group of readers will 
respond niost favorably to writing closest to the norm i,n a saniple--that the 
training in fact ruined their ability to road by defining the terms of success 
as relative to most of the writing in a limited number of samples. 

But a rh^toridUn, in fact Aristotlei suppliGd yet anothor theory that 
would account far the apparent inaccuracy of, and thus lack of trust of, those 
readers. Aristotle knew that we might often discover teachers grading what 
thay thought to be student writing lower if it omitted exemplification and made 
points without Illustrations to back them up. The Rhetoric (2,21), explicitly 
says that maxims-^broad genGralizations--are suitable evidence for speakers of 
mature years, but that young speakers should not use generalities where they 
lack experience. The readers in this study were accurate and consistent with 
a rhetoric-based reading theory^ the essential element omitted from a holistic 
judgment based on relative time and frequency* These readers responded as all 
readers do* on the basis of expectations about the writers, not exclusively* 
nor even primarily^ on the basis of expectations about a text. 

Reading is always a situational rather than text-centered act. Readers i 
like writers, make meaning, read to some purpose. Their mood, the recompence 
for reading--in general, their motives--w1 11 determine their responses to any 
text just as a writer's Intentions and sense of purpose will define the content, 
tones and voice of writing. Just as writers have more or less experience with a 
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variety of aims and itiodos of dlscoursei and thus my be skilled 1n one form, 
format, or audience but not anothers so also ard rGadors more or loss 
fatnlllar with partlculor sorts of writing and more or less used to bolng the 
audience for certain intontlons. They are prejudlclai 1y well or 111 pre- 
disposed to cortaln authors or groups of authors* And tholr faniiHarlty with 
the diction, syntaxi and historical or Invented G^ainplcs of the author will 
detdniUne thoir ability to idGntlfy with that author^ to construct the moaning 
that was Intended when composing* 

This description of the rQadlng procQSs does not doflnc "relative read- 
abiltty" as behavlorlsts would ^ as a matter of processing tifiie depending on 
given contexts. It does not say, as Blederich did , that readers are variously 
predisposed toward certain features of texts i Nor does it mean to suggest that 
readers are consistent or capable of classification according to one or another 
habitual reading pattern. Instead it places questions of quality in a com- 
plicated context of interaction between specific writers and specific readers 
who have many writing and reading processes* And neither Is this situational 
modpl of the reading process only a theoretical abstraction. It explains why 
-valuators want to know ''who wrote this?'' before they read as often as 
visitors ask the sex of a new baby before they speak to it. It explains why 
students who do poorly on writing samples may get high grades in courses, the 
grades of which measure the quality of relationship between reader and writeri 
not only the quality of an anonymous te^t* 

Even the most enlightened and helpful evaluators of student writing^ 
Lee O'Dell and Charles Cooper, suggest evaluative measures whose results are 
located only In behavioral domains rather than in complete descriptions of 
reading and writing* In a recent article in CCC 0*Dell suggests, for example, 

ERIC ^ 
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tdsting students wrHIng for nmo t\m one purpose in ofd^r to ddtorniInG 
whether the quality of writing 1s purpose-specific, lid suggests that "whereas 
we once could use a singlei widely agrGed-upon procGdurc for evaluating all 
the writing done In a given mode, we now have to use ti variety of cvalu- 
ativo procedures* [he favors Lloyd-Jones' priniary trait guides] i itmst of which 
we have to creatG for ourselvds," Ho continues by suggesting also the applica^ 
tion of diverse tools for evaluation, but he nowhere suggests that the rosults 
generated by these tools might depend on a complex set of changing attitudes 
on the part of feaders and students i While a writer niay, for Instance, niore 
proficiently writo a casual letter home than an essay to the general reader 
*^Vsuading against violence, a reader niay not be able to judge the effectlvo'- 
ness of the letter at all if that riador was not its real audience. 

I am calling additionally for methods much more congfuent with the 
processes of writing and of reading* A situational method would correlate 
results from current ineasurements with the energy and Intentional Ity of the 
acts of writing and reading. The motivation and the attitudes of both con- 
tributors to the meaning of a text would not be ignored. As a student, I knew 
my teachers valued improvement. I started slow, I knew that one teacher loved 
Strunk and White; I el1ni1nated ''the fact that," As a reader, I am prejudiced 
against the uses of "structure" as a verb and of small circles for the dots 
over 1 - s , I expect Russell Baker to funny, and I am willing to believe 
Bronowski^s arguments supported by fewer eKamples than I demand of students 
writing to make the same points. In other words, I havi written and read with 
varying degrees of authority and varying senses of not only my audiences' 
knowledge or prejudice about my subject, but also my audiences" knowledge and 
prejudices about me, I have evaluated my own work by how well it does all that 
I want It to do for my audience In a parttcular situation. 
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Sirtcd owf new evaluativG tools do not take into dccdunt those variablGS 
of wi^itlng and feadfng situdtionsi inuch less t\m particulate perfofniance fulds 
of slinuiations I which is what school and tost writings always aroi thGir re** 
suits are dangerous in the hands of those whose minds may be enipty of a sense 
of situation I of human intQraction. Wg do not now have a complete caldulus for 
evtiluation* and wg must ticknowlodgG first that we do not, Although the Invan- 
tory of tools we have grows, and seorns to bo growing in promising way§, wo aro 
not home yot. We would moxl to ddvlso naw ddscrlptions of the affective as 
wdll as cognitive aspects of writinfj and reading, account for the predispositions 
of writer and reader, and set such measurenionts in varying rations to the results 
from tools we now havo before we werGi If writing and teaching writing are to 
remain human act1v1tios but also become a coherent humanistic dlsclpllnoi we 
need an interpretive theory Informed by measurements of the variables In dynamic, 
organic situations. We mighty for Instancei investigate writers' ovaluations of 
their own writing, Their opinions of Its quQlity relative to their perceived 
abilities mayi especially 1n extremely poor and entremely proficient writers, be 
an accurate control on other judgments suggested by text-measuring tools, A 
receptivity measurement for readers that identified their motivations, their 
prior expectations about the writer, and their expectations about the features 
of an appropriate text*s response to a writing task might also be used to normal- 
ize other results. 

Whether or not we have such additional measurements, we must as least 
begin to distinguish, clearly and 1ns1stentlyi between descriptive measurements 
of texts and qualitative judgments of the success of writing. Making this dis- 
tinction Is the only way to prevent traditional attacks on the brief impact of 
the English Department's teaching. And it will be the only way to forestall new 
criticism that is bound to result from teaching toward changes in, for example, 
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wofda per t-unU that do not t^otilly cfcaiQ cluingo In writoi^'s ability to 
solve a ccmpHeated Wf1t1ng problem fr'otn the Inventive subtext to the edited 
"publication." 

The bottorn line for tQcichlng writing, ono t do not find nckhowlddfjod 
in itiost sdhool wflting and tosting proarams, Is, finally, that a good writftr 
controls & variety of discourse sltuatlonsj arid bas boon tniight not only a 
series of rules, but also tho stated and hidden ogendfls of ilny piirtlcular 
writing situation. Rhetoricians would teach not only what a reader wants and 
needs frojn a text, but also the situational varlabnity, oven within acadcnilc 
writing situations, of those deiMnds. Useful Justifications for what we mean 
when we say a student wrote either well or poorly aro aliead of us, and we 
should not rest now, with a new granmar, but without a cofflplete pragmatiGS, 
of interpretation. 



11 



